Papers with textual inference
A logical-based corpus for cross-lingual evaluation (D19-61)
Copied to clipboard
| Challenge: | a recent study shows that deep learning models can be used to solve textual inference tasks using simple linguistic patterns. |
| Approach: | They propose a set of syntactic tasks focused on contradiction detection that exploit linguistic patterns. |
| Outcome: | The proposed tasks can be implemented in English and Portuguese. |
Bag-of-Words Transfer: Non-Contextual Techniques for Multi-Task Learning (D19-61)
Copied to clipboard
| Challenge: | Existing approaches to multi-task learning take advantage of transfer among tasks . generative reconstruction of the observations is not included in the standard framework . |
| Approach: | They propose to use a syntactically-oblivious pooling encoder and pre-trained word embeddings to improve sentence-level representations. |
| Outcome: | The proposed techniques yield similar performance on a universe of task combinations while reducing training time and model size. |
SETI: Systematicity Evaluation of Textual Inference (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing pre-trained language models (PLMs) have shown remarkable performance on this task, but little is known about their ability to address compositional generalization. |
| Approach: | They propose a benchmark to evaluate pre-trained language models' systematicity in the domain of textual inference. |
| Outcome: | The proposed benchmark evaluates pre-trained language models on six widely used PLMs. |
Editing Factual Knowledge in Language Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | KnowledgeEditor can be used to edit factual knowledge stored in Language Models without the need for expensive retraining or fine-tuning. |
| Approach: | They propose a method which edits factual knowledge implicitly stored in Language Models and uses it to fix 'bugs' and 'obvious errors' they train a hyper-network with constrained optimization to modify a fact without affecting the rest of the knowledge; the hyper-netzwork is then used to predict the weight update at test time. |
| Outcome: | The proposed method can be used to edit factual knowledge without retraining or fine-tuning and can fix 'bugs' or unexpected predictions without the need for expensive re-training or meta-learning. |
Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic (2024.emnlp-main)
Copied to clipboard
Nathaniel Weir, Kate Sanders, Orion Weller, Shreya Sharma, Dongwei Jiang, Zhengping Jiang, Bhavana Dalvi Mishra, Oyvind Tafjord, Peter Jansen, Peter Clark, Benjamin Van Durme
| Challenge: | Recent language models allow structured reasoning with text, but lack of a clear protocol for discerning entailment causes noisy datasets and limited performance gains. |
| Approach: | They propose a consistent approach to annotating decompositional entailment and evaluate its impact on LLM-based textual inference. |
| Outcome: | The proposed approach has higher internal consistency than prior decompositional entailment datasets and significantly improves proof quality and accuracy. |